The present invention relates to a rectification apparatus of a stereo vision system and a method thereof and, more specifically, to a rectification apparatus of a stereo vision system and a method thereof, capable of simplifying a complicated matrix operation of calculating coordinates of a camera image corresponding to coordinates of a rectified image by substitution to a linear function.
Human vision is one of senses for obtaining surrounding environment information, and is to recognize a distance by synthesizing two kinds of visual information incoming through two eyes into one kind of information. A system obtained by embodying such a visual structure of human with two cameras is a stereo vision system.
The stereo vision system simultaneously receives left and right images from two cameras using two cameras imitating the human vision, and calculates a distance from the cameras to a subject from the left and right images, to obtain a 3-dimensional image.
Since two cameras are installed at spatially different positions, a disparity occurs between two cameras with respect to the same subject. In this case, according to phenomenon of binocular disparity, the disparity gets larger as the cameras get closer to the subject, and the disparity gets smaller as the cameras get farther from subject. The stereo vision system acquires a 3-dimensional distance information from disparity information based on such binocular disparity.
In the stereo vision system, the disparity information is calculated through a stereo matching process. In this case, the stereo matching is to find matching points corresponding to the same point of the subject in left and right images, and to calculate disparity information between the matching points found in the left and right images.
When the matching points are to be found in the stereo matching process, the operation thereof is performed for each pixel. In this case, since the stereo cameras physically have positional difference from each other, in order to find one point in a left image in a right image, a disparity searching process has to be performed on all points of the right image for each pixel. In such a case, since the entire image has to be searched to find the matching point for one pixel, the amount of operation is large, and thus there is a problem of difficulty in actual embodying.
Accordingly, since a real time image process and embodying of a 3-dimensional image are difficult in the stereo vision system due to a disparity searching time necessary in the stereo matching process, a rectification process of reducing a disparity search range to one dimension by reducing a search range for finding disparity to one row by allowing up and down disparity between left and right images to coincide with each other is necessary.
In the stereo vision system, two cameras are arranged on the same row through a rectification process, and it is possible to extract disparity information of an object on the basis thereof. In stereo cameras, tolerance may occur in a producing process or a position of the camera may deviate during using. In this case, the stereo camera may not be arranged on the same line. Particularly, in the case of a camera mounted on a vehicle, a position thereof easily deviates due to impact or the like. Accordingly, it is difficult to allow the row between the stereo cameras to physically coincide, and a rectification process of allowing the row to coincide is necessarily required.
FIG. 1A is a diagram illustrating a component of a stereo camera for explaining a rectification process of a stereo vision system according to the related art, FIG. 1B is a diagram illustrating left and right images photographed by a stereo camera, and FIG. 2 is a diagram sequentially illustrating the rectification process illustrated in FIG. 1A and FIG. 1B.
Referring to FIG. 1A, FIG. 1B, and FIG. 2, in a rectification process, a stereo vision system allows up and down disparity for left and right images of stereo cameras 11 and 12 to coincide, to reduce a disparity search range a in a stereo matching process to one dimension.
First, in the rectification process, coordinates of a rectified image plane are assumed (S1), and the coordinates of the rectified image plane are moved along a camera axis to calculate coordinates of the camera coordinate axis (S2).
Thereafter, in the rectification process, the image is rotated to the coordinates before the rectification on the camera coordinate axis, and the coordinates of the camera coordinate axis before the rectification are calculated (S3), and the coordinates calculated in Step (S3) are projected onto the camera image plane, and coordinates of the camera image plane are calculated (S4). Lastly, in the rectification process, pixel values of the camera image plane coordinates calculated in Step (S4) are mapped to pixel values of the rectified image plane coordinates assumed in Step (S1), (S5).
As described above, in the rectification process, a virtual rectified image is assumed, Steps (S1 to S5) are repeated from a point of coordinates (0, 0) of the virtual rectified image plane as much as camera resolution, camera image plane coordinates before performing the rectification are found, and the coordinates of the rectified image plane are gradually filled with a pixel value of the coordinates.
In this case, in the rectification process, it is possible to acquire processes of axial movement and rotation from the coordinates of the rectified image plane to the camera coordinates, and axial movement to the coordinates of each camera image plane, by using Equation 1.
                              [                                                    x                                                                    y                                                                    z                                              ]                =                  A          ·                      R            rect            T                    ·                      A            rect                          -              1                                ·                      [                                                                                x                    rect                                                                                                                    y                    rect                                                                                                1                                                      ]                                              [                  Equation          ⁢                                          ⁢          1                ]            
In Equation 1, [x y z]T denotes coordinates of a camera image homogeneous coordinate axis, A denotes intrinsic parameters of a camera as a matrix of 3×3, Arect denotes intrinsic parameters of a rectified image plane as a matrix of 3×3, Rrect denotes a rectification rotation matrix of 3×3, and [xrect yrect 1] denotes coordinates of a rectified image plane homogeneous coordinate axis. In Equation 1, the intrinsic parameters of the camera and the intrinsic parameter of the rectified image plane, and the rectification rotation matrix, which are the other parameters excluding the coordinate, are fixed parameter extracted through camera calibration.
As described above, in order to perform the stereo matching process in the stereo vision system, the rectification is a preprocessing process required necessarily, but the rectification process is formed of a complicated matrix operation such as axial movement and image rotation, and there is a problem of requiring a great amount of operations in spite of the preprocessing process.